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Abstract 

This  paper  gives  specific  computational  details  and  experience  with 
a  group  theoretic  integer  programming  algorithm.  Included  among  the 
subroutines  are  a  matrix  reduction  scheme  for  obtaining  group  represen- 
tations, network  algorithms  for  solving  group  optimization  problems,  and 
a  branch  and  bound  search  for  finding  optimal  integer  programming  solu- 
tions. The  innovative  subroutines  are  shown  to  be  efficient  to  compute 
and  effective  in  finding  good  integer  programming  solutions  and  providing 
strong  lower  bounds  for  the  branch  and  bound  search. 


6355r>4 


Introduction 

In  this  paper  we  report  on  computational  experience  with  a  new  integer 
programming  algorithm  (IPA).  Details  of  how  group  theory  can  be  used  to 
solve  IP  problems  are  discussed  in  Gorry  and  Shapiro  [12].  Reference  12 
also  contains  a  brief  discussion  of  some  preliminary  computational  experience, 
but  little  detail  was  given  there  about  computation  and  moreover,  some  of 
the  subroutines  in  the  preliminary  version  of  the  code  were  quite  primitive. 
The  current  version  of  IPA  contains  subroutines  which  are  fairly  tight  for 
code  written  in  basic  FORTRAN.  Our  primary  purpose  here  is  to  try  to  esta- 
blish that  group  theoretic  IP  methods  as  embodied  by  these  subroutines  are 
easy  to  construct  and  effective  to  use  in  solving  IP  problems.  An  important 
secondary  purpose  of  this  paper  is  to  report  on  computational  experience 
with  the  group  theoretic  constructs  discussed  abstractly  in  a  number  of  paoers 
(e.g..  Glover  [6,  7],  Gomory  [8,  9,  10],  Hu  [15,  16]). 

We  chose  to  implement  IPA  in  FORTRAN  because  we  desired  to  have  the 
ability  to  transfer  the  system  from  one  computer  to  another.  We  have  found 
this  ability  particularly  useful,  and  to  date  the  system  has  been  implemented 
on  various  versions  of  the  IBM  360  System  (Models  65,  67,  75  and  85)  and  on 
the  Univac  1108.  We  have  made  the  more  obvious  attempts  to  make  efficient 
the  encoding  of  this  system  in  that  we  have  been  concerned  with  data  struc- 
tures and  sequencing  of  operations  within  the  system.  On  the  other  hand, 
we  have  not  implemented  any  of  this  system  in  assembly  language  for  a  given 
machine.  We  do  feel  that  such  an  implementation  would  significantly  improve 
the  computational  efficiency  of  our  system  but  the  transferability  of  the 
FORTRAN  had  definite  appeal  for  our  purposes  in  the  past. 


Overview  of  IPA 

We  begin  with  a  statement  of  the   IP  problem  in  its   initial    form.     This 
problem  is 
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mm  z  =     I     c  .w. 


n+m 
s.t.     I     a.  .w.  =  b. 
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TJ     J  T 


i  =  l ,...  ,m 


(1) 


w.   =  0,1,2,... 

w.   =  0  or  1 
J 


J   e   U 
J   c   1 


where  the  coefficients  c,   a..,  b.   are  integers,   U  and  Z  are  disjoint  subsf^ts 
of  {1,2,... ,n+m} ,   and  U  U  Z  =   {1 ,2,. . . ,n+m}.     Let  a .  denote  a  generic  m 
vector  with  components  a..,   and  let  b  be  the  m  vector  with  components  b . . 
Problem  (1)   is   solved  first  as   a  linear  programming  (LP)   problem  by  a  simplex 
algorithm.     Without  loss  of  generality,  assume  the  optimal   LP  basic  variables 
are  y.   =  w  ^.,   i=l,...,m,   and  the  non-basic  variables  are  x.   =  w.,  j=l,...,n. 
Let  U„  and  Zg  be  the  partition  of  the  index  set   {l,...,m}   into  the  index  sets 
of  unconstrained  and  zero-one  basic  variables;    let  Uj^  and  Zj^  be  similarly 
defined  for  the  non-basic   index  set   fl,...,n}. 
The  optimal    LP  basis  B 
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m,n+l 


i  ,n+m 


m,n+m 
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is   used  to   transform  (1)   to  the  equivalent  form 


min  z  =  Zp,  +     z     c.x.  (3a) 


s.t.  y.  =  b.   -     Z     a..x.  i=l,...,m  (3b) 

y^-  -  0,1,2,...  i  t   Ug  (3c) 

y^.  =  0  or  1  i  e  Zg  (3d) 

Xj  =  0,1,2,...  j  c  U^  (3e) 

Xj  =  0  or  1  J  ^   ^N  ^^^^ 

where  c.   >  0,  j  =  l,...,n,  b.   >  0,  i=l,...,m,  b.   <  1,   i   e  Z„. 

J     —                                                             1     —  1     —  D 


The  optimal    LP  solution   is   x^  =  0,  j=l,...,n,  y^  =  b.,   i=l,...,m. 
Clearly,  the  original    IP  problem  (1)   is  solved  if  y^  is   integer  for  all    i. 
If  one  or  more  basic  variables    is     not  integer  in  the  optimal   LP  solution, 
then  some  of  the  independent  non-basic  variables   x.   must  be  set  at  positive 
integer  values  in  order  to  make  the  dependent  basic  variables   integer  and 
non-negative.     We  call    a  non-negative  integer  vector  x  =   (xi,...,x  )  satis- 
fying (3e)   and   (3f)   a  correction  to  the  optimal   LP  solution.      If  the  re- 
sulting y.    given  by   (3b)    satisfy   (3c)   and   (3d),   then   x   is   a   feasible  correction, 
The  least  cost  feasible  correction  is  an  optimal   or  minimal   correction. 

Problem  (3)    is   transformed  to  another  equivalent  problem  as   follows. 
Let  Z       denote  the  cyclic  group  of  order  q..     Each  non-basic  activity  a.   is 

mapped  into  an  r-tuple  of  integers  a.   =   (a,  .  ,ap^. ,. . .  ,a   .)  where  a.,  c  1 

r 
and  the  integer  q.   satisfy  q.  ^2,  q. |q.^, ,   n     q.   =  D  =    |det  B | .     Similarly, 

the  right  hand  side  vector  b  is  mapped  into  the  r-tuple  of  integers 


e  =   (3i,...,6   )   where  g.    e   Z     .      The  collection  of  D  r-tuples  of  the  form 

(a,  , .. .  ,A   ),  A.    e  Z      , under  addition  modulo   (q-,,...,q   )    form  an   abelian   group 

and  hence  these  methods  are  called  group  theoretic.     The  equivalent  formula- 
tion to  (1 )   and   (3)   is 


min  z  =  z^  +     z     c.x.  (4a) 


n     _ 

s.t.  y.   =  b.   -     z     a..x.  i=l,...,m  (4b) 

•'t         1       j^l     ij  J  ^     ' 

n 

Za..x.    .:    6.(mod  q.)  i  =  l,...,r  (4c) 

y^    >  0,  i    e   Ug  (4d) 

y.  1  1  i   E  Zg  (4e) 

Xj  =  0,1,2,...  j   e  U^  (4f) 

Xj   =  0  or  1  J   ^  ^N  ^^3^ 


Problem  (4)   is  the  starting  point  for  the  group  theoretic  approach.     It  is 
difficult  to  explicitly  consider  all    the  constraints   in   (4),   and   IPA  pro- 
gresses by  ignoring  certain  constraints   and  solving  the   related  problems 
which   result.      The  optimal    solutions   to  these  problems   are  then   used  in   a 
manner  to  be  described   later  to   find  an  optimal    solution  to   (4).     We  simply 
mention  here  that  this  method  of  ignoring  some  constraints   thereby  creating 
easier  optimization  problems   is   called  relaxation  (see   [5],   [i2]>   Cls])- 
Specifically,   if  X  is  the  set  of  feasible  corrections   in   (4),  then  the  al- 
gorithm will    usually  deal   with   sets   X'    satisfying  XCX'.      Then,   if  a 


n 

correction  x*  which  is  optimal  in  E  c.x.  subject  to  x  e  X'  is  feasible  in 

i=l  ^  -^ 

n  _      n  _ 

(4),  (i.e.,  X*  e  X),  then  x*  is  optimal  in  (4).  Moreover,  e  c-x^  <_  E  c.x. 

j=l  J  J   j=l  ^  ^ 

for  all  feasible  corrections  x  e  X.  These  properties  of  relaxation  are 
the  basis  for  the  branch  and  bound  search  of  IPA,  and  also  for  the  data 
manipulation  schemes. 

Table  1  gives  timings  of  IPA  of  some  real  life  IP  problems.  We  will 
refer  to  these  problems  at  various  points  during  the  paper.  The  313  row 
airline  crew  scheduling  problem  was  solved  to  approximately  the  same  degree  of 
optimal ity  by  the   Ophelie  Mixte  code  in  64.15  seconds  on  the  CDC  6600. 
The  IBM  test  problem  was  solved  by  Geoffrion's  improved  implicit  enumera- 
tion code  in  114  seconds  on  the  IBM  7044. 

A  simplified  flow  chart  of  the  IPA  algorithm  is  presented  in  Figure  1. 
The  major  components  of  the  system  are  indicated  by  boxes  labelled  with  the 
names  of  the  principal  system  subroutines  (e.g.  LP  for  linear  programming, 
etc.).  In  what  follows,  we  will  discuss  each  component  briefly  and  present 
some  computational  experience  with  that  portion  of  the  system. 


Table  1 
EXAMPLES  OF  RESULTS  WITH  IPA 


OBLEM  TYPE 

No.    of 
Rows 

No.    of 
Variables 

LP 
Time 

Group 
Rep .   Ti  me 

Group 
Soln.   Time 

Search 
Time 

Total 
Time  ^ 

Machine 

DIA 
SELECTION 

12 

116 

1.34 

0.37 

0.19 

0.14 

14.0 

UNIVAC   1108 

XED  CHARGE 

14 

32 

0.07 

0.03 

1.70 

0.27 

2.36 

360/85 

PITAL 
INVESTMENT 

36 

72 

5.56 

0.91 

1.16 

103.2 

112.1 

360/67 

RSONNEL 
SCHEDULING 

57 

132 

6.86 

- 

- 

2.85 

33.0 

UNIVAC   1108 

'\ND 
ALLOCATION 

86 

195 

12.82 

1.52 

0.95 

2.59 

29.0 

UNIVAC   1108 

RLINE   CREW 
SCHEDULING^ 

313 

482 

35.14* 

5.10 

0.05 

150.0 

192.5 

360/85 

RLINE   CREW 
SCHEDULING 

176 

2385 

49.26* 

17.84 

0.01 

-0- 

67.11 

360/85 

^ER  ROLL 
SLITTING 

5 

54 

0.35 

0.06 

0.01 

0.0 

0.42 

360/85 

REEN   LOOM 
:UTTING 
PATTERN 
SELECTION 

27 

641 

4.38* 

C.72 

0.03 

0.13 

5.26 

360/85 

,1)     APPL. 
JNKNOWN 

26 

383 

10.08* 

0.56 

2.03 

123.07 

135.74 

360/85 

^  TEST 
PROBLEM 

50 

65 

0.39* 

0.06 

0.03 

3.36 

3.84 

360/85 

Dtal  Time  is  in  seconds. 


sasible  integer  solution  with  a  cost  within  3.5  percent  of  the  minimal  LP  cost  found 
/  search  in  time  indicated.  Optimal ity  was  not  proven,  run  was  terminated  with  a 
aximum  time  cutoff. 

P  time  for  IBM  360  MPS. 


(begin) 


READ  DATA 


SOLVE  LP  PROBLEM 
AS  LP  PROBLEM 


LP 


CONVERT  TO  GROUP 
OPTIMIZATION  PROBLEM 


SOLVE  GROUP 
OPTIMIZATION  PROBLEM 


GETREP 


UGP 
or  ZOG 


BRANCH  AND 
BOUND  SEARCH 


SEARCH 


-^ 


(done) 


Figure  1 


IPA  Linear  Programming 

IPA  contains  its  own  LP  routine  which  can  solve  most  LP  problems  with 
200  rows  or  less,  and  a  reasonable  number  of  variables.  We  put  less  time 
and  effort  into  this  LP  routine  relative  to  that  devoted  to  the  more  novel 
routines  discussed  below.  The  LP  routine  is  an  upper  bounding  variant  of 
the  simplex  method  written  in  FORTRAN  IV.  It  uses  single  precision  floating 
point  arithmetic  and  therefore  round-off  problems  can  develop,  especially 
if  the  determinant  of  the  LP  basis  becomes  large.  Since  large  basis  deter- 
minants make  the  group  theoretic  methods  difficult  to  use,  and  moreover, 
since  measures  are  used  to  try  to  prevent  large  determinants  from  occurring, 
the  round-off  problem  is  avoided  and  controlled  to  a  certain  extent.  The 
procedures  for  controlling  the  determinants  are  discussed  briefly  below  and 
in  more  detail  in  [13]. 

Nevertheless,  round-off  has  sometimes  been  a  problem  for  the  LP  routine 
and  in  these  cases  it  was  necessary  to  read  in  an  optimal  LP  basis  obtained 
from  another  system,  e.g.  the  IBM  360  MPS  linear  programming  code.  We  remark 
that  it  is  not  possible  to  use  the  final  (optimal)  LP  tableau  found  by  MPS 
because  MPS  does  not  give  the  determinant  of  the  optimal  LP  basis  from  which 
the  tableau  is  derived.  Without  knowledge  of  the  determinant,  the  transfor- 
mation to  group  optimization  problems  cannot  be  made. 


The  Group  Representational  Algorithm 

The  group  representational  algorithm  (GETREP)  is  based  on  a  method  due 
to  J.  H.  S.  Smith  for  characterizing  integer  solutions  to  a  system  of  linear 
equations  by  diagonalizing  integer  basis  matrices.  The  original  references 
are  [20]  and  [21].  Wolsey  [24]  shows  how  Smith's  method  is  used  in  IP; 
other  discussions  can  be  found  in  the  paper  by  D.  A.  Smith  [19]  and  the  book 
by  T.  C.  Hu  [16j.  We  remark  that  Cabay  in  reference  [1]  uses  similar  con- 
gruential  methods  to  find  exact  continuous  solutions  to  systems  of  linear 
equations. 

Conceptually,  the  procedure  beings  with  the  system  of  congruences 


n 

I  a^.x.  ;;  b.(mod  1)        i  =  l,...,m,  (5) 

j  =  l 


TJ  J     1 


which  is  equivalent  to  the  requirement  that  the  independent  non-basic  vari- 
ables X.   be  chosen  in  such  a  way  that  the  dependent  basic  variables  y.   are 
integer.      The   subroutine  GETREP  transforms    (5)    to   the  system  of  congruences 
(4c)  which  we  rewrite  here 


Y.     a..x.    E   6-(mod  q.)  i  =  l,...,r. 

j=l      ^J   J  ^  ^ 


The  reduction  is  necessary  for  efficient  computation  as  r  is  usually  on 
the  order  of  1  to  5  regardless  of  the  value  of  m. 

The  reduction  of  the  system  of  congruences  is  achieved  by  reduction 
of  the  m  X  m  matrix  F  =  fj^^.  =  D{a|^  ^^^  -  [l^   n+i-'^'  1^"^  >•  •  •  'i^'  '"''^  '•  •  •  '"• 
Through  a  series  of  passes  through  F,  the  matrix  is  diagonalized.  Each  of 
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these  passes  consists  of  finding  the  minimum  non-zero  element  in  F  and 
using  the  row  and  column  in  which  that  element  exists   for  elementary  row 
and  column  operations  on  F.     These  row   (column)  operations  are  like  the 
typical   Gauss- Jordan  reduction  steps  except  that  the  constant  multiple  of 
a  row  (column)  which  is   to  be  subtracted  from  another  row  (column)   is   deter- 
mined in   integer  arithmetic  and  the  reductions   are  made  modulo  D.      In  the 
event  that  a  pass   through  the  matrix  leaves  a  given  row  and  column  with  the  only 
non-zero  element  at  their  intersection,  the  row  and  column  are  in  general 
removed  from  any  further  consideration  in  the  di agonal izati on  procedure. 
The  only  exception  to  this   is  when  the  non-zero  intersection  element  does 
not  divide  the  determinant  or  does  not  divide  the  similar  preceding  element. 
In  such  a  case  the  representational   algorithm  "backs   up"   and  continues  the 
process.     We  will   omit  here  details  of  this  last  maneuver.     Basically,   the 
purpose  for  it  is  to  insure  that  the  minimal   group  representation  is  obtained 
by  the  algorithm.     The  diagonalization  procedure  is  complete  when  the  product 
of  the  non-zero  intersection  element  equals  the  determinant  of  the  basis 
matrix. 

If  the  diagonalization  procedure  has  revealed  r  subgroups   (that  is, 
if  r  elements  were  required  to  form  a  product  equal   to  the  determinant), 
then  the  a's  are  determined  by  the  first  r  inner  products  of  the  rows  of  a 
matrix  C  generated  from  the  elementary  column  operations  of  diagonalization 
procedure  and  the  non-basic  columns  taken  modulo  the  corresponding  subgroup 
orders. 
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Table  2a.  Group  Representational  Algorithm:  Computational  Experience 
(♦denotes  IBM  360/67;  all  others  Univac  1108) 


Number 

of 

Rows 

m 

Determinant 
D 

Number 

of 

Subgroups 

r 

Time 

in 

Seconds 

t 

Number 

of 

Non  Basics 

n 

1 

6 

180 

2 

0.90 

240 

2 

6 

1080 

3 

0.92 

240 

3 

•6  ■' 

10 

1 

1.13 

300 

*4 

9 

128 

3 

0.06 

14 

5 

12 

5832 

6 

0.35 

104 

6 

12 

14400 

5 

0.38 

104 

7 

18 

81600 

3 

1.14 

240 

8 

18 

55296 

5 

1.15 

240 

9 

18 

900 

2 

1.11 

240 

10 

18 

1158 

1 

1.01 

240 

11 

18 

960 

2 

1.04 

240 

12 

18 

23328 

4 

1.14 

240 

13 

18 

96 

2 

1.05 

239 

*14 

36 

384 

4 

0.91 

36 

*15 

36 

144 

2 

0.73 

36 

*16 

46 

420 

1 

3.41 

115 

*17 

50 

21744 

1 

2.19 

115 

*18 

50 

36288 

1 

2.90 

115 

19 

86 

280 

2 

1.46 

109 

20 

86 

140 

1 

1.40 

109 

21 

86 

1080 

1 

1.82 

109 

22 

86 

17100 

2 

1.53 

109 

23 

86 

540 

2 

1.37 

109 

24 

86 

380 

1 

1.60 

109 

25 

91 

2048 

3 

1.67 

104 

26 

91 

512 

3 

1.45 

104 
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Table  2b.  Group  Representational  Algorithm: 
Computational  Experience 

(IBM  360/85) 


Number 

of 

Rows 

m 

Number 

of 

Non  basics 

n 

Determinant 
D 

Number 

of 

Subgroups 

r 

Time 

in 
Seconds 
t 

1 

9 

12 

64 

2 

0.02 

2 

9 

14 

128 

3 

0.00+ 

3 

11 

11 

384 

2 

0.03 

4 

14 

18 

5025 

1 

0.03 

5 

36 

36 

864 

4 

0.12 

6 

36 

36 

1152 

5 

0.14 

7 

36 

36 

5760 

5 

0.15 

8 

36 

36 

6912 

5 

0.14 

9 

40 

58 

2 

1 

0.07 

10 

50 

15 

36 

1 

0.06 

11 

313 

169 

48 

2 

5.10 
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In  Tables  2a     and  2b  we  have  presented  some  representative  times   for 
the  algorithm  on  a  variety  of  integer  programming  problems.     As   can  be  seen 
from  this   figure,   in  spite  of  the  fact  that  the  algorithm  is  written  in 
FORTRAN,   the  computational   requirements  to  obtain  group  representations  are 
relatively  small.     We  have  not  performed  any  detailed  study  on  the  relation- 
ship between  the  computational   time  required  for  the  representational   al- 
gorithm and  such  variables  as  the  number  of  rows   in  the  problem  or  the  num- 
ber of  non-basic  columns.     There  are,   however,   certain  general   observations 
we  can  make  based  on  the  data  presented.      In  general   the  computational    re- 
quirement increased  with  the  number  of  rows   in  the  integer  programming  prob- 
lem because  the  number  of  elements   in  the  matrix  F  which  must  be  considered 
is  equal   to  the  square  of  the  number  of  rows.     Thus,   increasing  the  number 
of  rows  increases  the  amount  of  computation  required  to  diagonal ize 

the  fractional   matrix. 

On  the  other  hand,  our  experience  to  date  has  not 
shown  the  number  of  subgroups   to  be  an  important  factor  in  determining  the 
computational   time.     The  reason  for  this  is  that  a  single  pass  through  the 
fractional   matrix  will    produce  many  zero  elements  in  that  matrix.     Subse- 
quent passes  through  the  matrix  will   then  generally  find  a  number  of  constant 
multiples   for  rows  and  columns  which  are  zero  and  hence  effectively  can  be 
ignored.      The  number  of  non-basic   columns   in   the  problem  is   also  an   important 
determinant  of  the  computational    time.     Once  C       has  been  obtained,   the  cal- 
culation of  the  group  identities   for  the  non-basics   requires   the  multiplica- 
tion of  some  number  of  rows  of  C       be  each  of  the  non-basic   columns   in   turn. 
As   a   result,    if  the  number  of  non-basics   is  quite   large,   then   the  computa- 
tional   time   for  the   representational    algorithm  wil 1    also  be   relatively   large. 
Notice,  however,   that  these  times  will   show  only  a  relatively  large  increase. 


14 

In  other  words,  the  absolute  time  required  to  obtain  the  representation  is 
in  general  quite  small  and  for  the  problems  which  we  have  dealt  with  where 
the  number  of  rows  and  columns  is  less  than  say  300,  this  time  is  on  the 
order  of  one  to  three  seconds. 

In  spite  of  the  promising  computational  experience  with  GETREP,  further 
research  is  required  into  group  representational  theory  and  IP.  If  many 
group  problems  are  formulated  and  used  during  the  branch  and  bound  search, 
then  faster  group  representation. times  are  desirable.  Johnson  [14]  has  had 
success  with  a  fast  method  of  approximating  the  groups  as  cyclic  groups. 

Research  is  also  required  to  gain  a  better  understanding  of  the  re- 
lationship between  a  group  structure  and  the  IP  problem  from  which  it  is 
derived.  For  example,  changing  one  number  in  the  basis  B  can  radically 
change  the  group  structure.  Useful  structural  relationships  may  also  be 
identifiable  between  subgroups  in  the  representation  and  subsystems  of  the 
IP  problems.  Tompkins  and  Unqer  [23]  have  done  this  for  the  fixed  charge  problem. 
Finally,  it  is  interesting  to  note  that  Gomory  originally  expected  the 
groups  to  be  cyclic  in  most  cases  ([8;  p.  279]),  but  computation  thus  far 
is  counter  to  that  expectation. 

When  GETREP  is  finished,  it  has  produced  the  group  representations  of 
the  non-basics  (the  a.'s)  and  that  for  b  (denoted  by  e).  IPA  can  now  solve 
the  implied  shortest  route  problem  as  discussed  in  the  next  section. 
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Group  Optimization  Problems 

After  GETREP  has  obtained  the  group  representation  (4c),  IPA  solves 
one  of  two  group  optimization  problems.  These  problems  are  constructed 
by  ignoring  the  non-negativity  constraints  on  the  basics  and  possibly 
some  upper  bound  constraints.  As  mentioned  in  the  overview,  this  approach 
will  provide  either  an  optimal  correction  in  (4),  or  the  necessary 
information  about  lower  bounds  and  trial  corrections  from  which  to  do  a 
search  for  an  optimal  correction. 

One  group  optimization  problem  is  the  unconstrained  group  problem  (UGP): 

For  k  =  0,1 D-1  ,  find 

n 
G(X.  )  =  min  E  c  .x. 
K       j=l  J  J 
n 
s.t.  I   a..x.  =  A|^.{mod  q . ) ,  i  =  1  ,. . .  ,r  (6) 

X.  =  0,1  ,2,,..;  j  =  1  ,...,n 

This  problem  is  also  referred  to  as  the  asymptotic  problem  by  Gomory  [9]. 

Problem  (6)  is  a  shortest  route  problem  (see  [12], [17])  in  a  network 
with  D  nodes,  one  for  each  r-triple  X,  ,  directed  arcs  (A.  -  a-,X|^), 
j  -   1  ,... ,n,  drawn  to  every  node  X.  except  the  node  Xq  =  0.  The  arc 
length  or  cost  associated  with  an  arc  type  j  is  c.. 

The  unconstrained  group  problem  solves  (6)  for  all  right  hand  sides 
X.  rather  than  the  specific  right  hand  side  B  in  problem  (4)  because  the 
additional  paths  are  important  if  a  search  is  required.  An  optimal  solution 
x(X.  )  to  (6)  with  right  hand  side  Xj^  is  extracted  from  the  shortest  route 
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path  by  letting  x.(a,  )  equal  the  number  of  times  an  arc  j  is  used  in  the 
optimal  path. 

The  other  group  optimization  problem  which  IPA  sometimes  uses  is  prob- 
lem (6)  with  additional  upper  bound  constraints  (often  upper  bounds  of  one) 
on  some  of  the  non-basic  variables.  This  problem  is  called  ZOG.  The  al- 
gorithm for  solving  this  problem  is  given  in  reference  [12].  Consistent 
computational  times  for  it  are  not  available  since  it  has  undergone  several 
changes.  Indications  are,  however,  that  it  is  only  slightly  slower  than 
UGP  for  problems  with  basis  determinants  of  less  than  1000  although  it  does 
require  more  core  storage  than  UGP. 

Tables  3a  and  3b  give  times  for  the  solution  of  (6)  using  the  uncon- 
strained group  problem  algorithm  of  [12];  this  algorithm  was  originally 
proposed  in  [17].  The  algorithm  finds  in  a  few  seconds  the  entire  short- 
est route  tree  for  IP  problems  with  basis  determinants  of  up  to  5000.  The 

determinant  is  the  primary  factor  in  determining  the  computation  time 
of  UGP  but  the  number  of  non-basics  n  and  the  difficulty  of  the  problem 
also  are  important  factors.  Notice  also  that  the  time  appears  to  increase 
linearly  in  these  factors. 

There  are  three  other  algorithms  in  the  literature  for  solving 
group  optimization  problems.  The  first  was  the  algorithm  proposed  by 
Goromy  in  [9]  in  which  the  group  optimization  problem  (6)  was  originally 
formulated.  Published  computational  experience  with  this  algorithm 
does  not  appear  to  exist. 

A  second  algorithm  for  solving  group  optimization  problems  is  given 
by  Glover  in  [6]  which  includes  some  computation  times  for  randomly 
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generated  group  optimization  problems.  It  is  difficult  to  make 

comparisons  between  Glover's  results  and  our  own  because  the  characteristics 

he  gives  for  randomly  generated  group  optimization  problems  are  different 

from  the  ones  from  actual  IP  problems. 

Specifically,  for  group  problems  derived  from  real  IP  problems  the  a- 

in  (6)  do  not  appear  to  be  randomly  generated  and  the  range  of  arc  costs 

is  definitely  greater  than  the  range  given  by  Glover.  These  characterist--cs 

of  group  optimization  problems  derived  from  real  IP  problems  should 

make  them  easier  to  solve  than  randomly  generated  ones  because  a 

relatively  small  number  of  non-basic  activities  with  low  costs  dominate 

the  shortest  route  solutions. 

Finally,  there  is  the  algorithm  of  Hu  [15]  which  is  an  adaptation 
of  the  shortest  route  algorithm  of  Dijkstra  [2].  The  essential 
difference  between  Hu's  algorithm  and  UGP  is  the  manner  of  selecting 
a  path  in  the  network  to  extend.  Hu's  algorithm  searches  through  all 
the  paths  for  which  extension  is  indicated  to  find  the  minimal  cost 
one,  whereas  UGP  selects  the  first  such  path  it  can  find.  We  have 
experimented  briefly  with  the  Dijkstra-Hu  approach  by  making  the 
appropriate  change  in  UGP;  it  appears  that  this  change  causes  the  time 
to  increase  by  as  much  as  a  factor  of  five  to  ten  (Hu  does  not  give  any 
computational  experience  in  [15]).  The  explanation  of  why  the  Dijkstra 
method  might  be  ineffective  in  this  case  is  because  the  group  shortest  route 
network  has  special  structure;  namely,  the  same  types  of  arcs  with  the 
same  arc  costs  are  drawn  to  every  node.  Thus,  low  cost 
dominating  arcs  are  discovered  quickly  and  can  be  used  repeatedly. 
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Table  3b.  Solution  of  the  Unconstrained 

Group  Optimization  Problem: 

Computational  Experience 

(IBM  360/85) 


Number 

of 

Non  Basics 

n 

Determinant 
D 

Number 
of 
Subgroups 
r 

Time 

in 

Seconds 

t 

1 

58 

2 

1 

0.00+ 

2 

195 

48 

2 

0.05 

3 

12 

64 

2 

0.01 

4 

14 

128 

3 

0.03 

5 

11 

384 

2 

0.05 

6 

36 

864 

4 

0.21 

7 

36 

1152 

5 

0.29 

8 

18 

5025 

1 

1.70 

9 

36 

5760 

5 

3.74 

10 

36 

6912 

5 

1.55 
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Johnson  [14]  has  developed  a  special  algorithm  to  solve  cyclic  group 
problems  quickly  where  the  cyclic  groups  are  approximations  to  subgroups 
of  the  true  group  implied  by  the  matrix  B. 

We  now  turn  to  the  question  of  when  the  group  problem  or  asymptotic 
problem  (6)  does  in  fact  solve  the  IP  problem  (4)  from  which  it  was  derived. 
In  reference  [16]  on  page  347,  T.  C.  Hu  states,  "Although  we  have  a  suffi- 
cient condition  that  tells  when  the  asymptotic  algorithm  works,  actual 
computation  will  reveal  that  the  algorithm  works  most  of  the  time  even  if 
the  sufficient  condition  is  not  satisfied."  Computation  has  in  fact  re- 
vealed that  solving  the  asymptotic  problem  does  not  solve  the  original 
problem  for  most  real  life  IP  problems  of  any  size,  say  problems  with  more 
than  40  rows.  The  positive  search  times  on  all  but  two  of  the  problems  in 
Table  1  indicates  that  the  solutions  to  the  asymptotic  problems  (6)  were 
infeasible  in  the  original  problems.  The  difficulty  is  compounded  by  the 
fact  that  problem  (6)  almost  always  has  many  alternative  optimal  solutions, 
only  one  of  which  may  be  feasible  and  therefore  optimal  in  (4).  This  was 
the  case,  for  example,  for  the  57  row  clerk  scheduling  problem.  The  op- 
timal IP  correction  was  optimal  in  (6)  but  it  had  to  be  found  during 
search.  The  asymptotic  problem  does  provide  good  lower  bounds  in  objective 
function  costs  and  good  candidate  solutions  for  the  search  which  is  discussed 
in  the  next  section. 
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Search  Procedures  in   IPA 

The  branch  and  bound  search  performed  by  IPA  implicitly  considers  all 

non-negative  integer  corrections  x  in  (4).     At  an  intermediate  point  of 

computation,  the  minimal   cost  solution  found  thus  far  is  called  the 

incumbent  and  denoted  by  9.  with  incumbent  cost  z  =  Zq  +  ex.     The  algorithm 

generates   IP  subproblems  of  the  following  form  from  explicitly  enumerated 

non-negative  integer  vectors  x 

j(x)_ 
z(x)  =  Zo  +  ex  +  min     I      c.x  (7a) 

j=l       ^  ^ 
j(x)  n 

s.t.     Z     a..x.   <b-       E  a..x.       i=l,...,m  (7b) 

j=1     ^  ^  ~  j=j(x)       ^^  ^ 

j(x)  n  ,      .       . 

1=1,. ..,r  (7c) 

(7d) 
(7e) 
(7f) 
(7g) 

where  j(x)  is  either  the  smallest  index  j  such  that  x.>0  if  this  index 
corresponds  to  an  unbounded  variable,  or  it  is  the  smallest  index  minus  1 
such  that  x.>0  if  this  index  corresponds  to  a  zero-one  variable.     The 
summations  in   (7)  are  restricted  to  the  range  1   to  j(x)  in  order  that  the 
search  be  non-redundant. 

If  we  can  find  an  optimal   solution  to   (7),  then  we  have  effectively 
tested  all   non-negative  integer  corrections  x  ^x,  x.  =  x.,  j  =  j(x)  +  l,...,n. 


Z     a..x.   =  &.    -       Z         a..x.   1 
j=l      '^   ^         '        j=j(x)     '^   ^ 

^mod  q. 

y.    >0             ieUg 

^il^             '^h 

y.   =  0,1,2...         jeU^n{l,2,.. 

.,j(x)} 

X.  =  0  or  1             JeZ|^n{l  ,2,.. 

.,j(x)) 
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and  they  do  not  have  to  be  explicitly  enumerated.  The  same  conclusion  is 
true  if  we  can  ascertain  that  z(x)  >_  z  without  discovering  the  precise  value 
of  z(x).  If  either  of  these  two  cases  obtain,  then  we  say  that  x  has  been 
fathomed.  If  x  is  not  fathomed,  then  we  abandon  (7)  and  create  new  sub- 
problems  of  the  same  form  as  (7)  from  the  solutions  x  +  e.,  j  =  l,...,j(x), 
where  e.  is  the  j   unit  vector. 

We  try  to  fathom  (7)  in  the  same  manner  that  we  tried  to  solve  (4); 
namely,  by  ignoring  some  of  the  constraints.  If  problem  (6)  was  the 
group  optimization  problem  selected  before  search  began,  then  we  use  the 
group  optimization  problem 

n  n 

G(3  -  E  a.T. )  =  min  Z  c  .x  . 
j=l  J  J       j-1  J  ^ 

n  "     ^ 

s.t.   Za.  .X.  =  3-  -   Z   a.  .'x.(mod  q. )  i  =  l,...,r       (8) 
j=1^^  ^    '   j=j(x)  'J  J     ' 

X.  =   0.1.2,...     j  =  l,...,n 

The  use  of  (8)    "in  trying  to  fathom  (7)  is  given  in  Figure  2.     Here  we  will 
comment  briefly  on  the  various  components  of  the  search. 

Our  discussion  of  the  search  will   assume  the  use  of    UGP;  at  the  user's 
choice,  ZOG  can  be  used. 

In  a  problem  with  zero-one  variables,  the  latter  will  yield  better  bounds 
for  the  fathoming  test.     The  basic  generation  of  the  search,  however,  would 
be  the  same  as  with  UGP. 

The  solution  to  the  group  problem  chosen  is  maintained  in  core  in  a  list- 
structured  form  which  permits  the  rapid  retrieval   of  backtracked  solutions  to 
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(8).  Basically,  for  each  node  of  the  group  problem,  the  system  stores 
the  cost  of  the  shortest-route  path  to  the  node,  the  identity  of  the  arc 
into  the  node,  and  a  pointer  which  can  be  used  to  reconstruct  the  whole 
path  back  to  zero  in  the  group  network. 

The  remaining  free  core  memory  is  used  by  the  search  routine  for 
the  storage  of  subproblems.  The  space  is  used  as  a  buffer  area  in  which 
blocks  of  subproblems  for  consideration  are  stored.  As  the  search  generates 
more  subproblems,  blocks  of  them  are  moved  to  disk  storage,  generally  to 
be  returned  to  core  later  when  the  space  becomes  available.  Through  the  use 
of  secondary  storage,  the  system  is  able  to  maintain  a  very  large  number  of 
active  subproblems  while  it  searches  for  an  incumbent. 

Step  1 :  Here  a  subproblem  is  chosen  for  the  fathoming  tests.  In  the  curr'ent 

system,  subproblems  are  considered  in  the  order  of  their  generation.  For 

each  subproblem,  the  system  stores:  1)  the  bound  obtained  when  the  subproblem 

n  ^ 
was  generated;  2)  the  index  of  the  node  3(x)  =  3  -  T.a.x-   for  the  particular 

x"^  in  question;  3)  x  itself  (with  only  the  non-zero  elements  stored  as  pairs 

(J^fX.  ));  and  4)  the  shortest  route  path  from  B(x)  to  0. 


'1 

n 
Step  2:  If  the  bound  zo  +  G(6  -  Z  a.x.)  is  greater  than  or  equal  to  the 

j=l  ^  ^ 
current  incumbent  cost  2   then  the  subproblem  is  deleted,  because  it  is 

fathomed  by  bound.  As  will  be  seen  below,  this  is  the  second  time  this  subproblem 

has  been  tested  for  bound.  The  first  time  was  when  it  was  created.  Because 

the  subproblem  is  still  active,  we  know  the  first  check  failed  to  fathom  it. 

In  general,  a  very  large  number  of  other  subproblems  have  been  considered 
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since  then,  and  a  new  incumbent  with  lower  cost  may  have  been  found.     Hence 
this   second  test  may  fathom  the  correction. 

If  the  subproblem  is  not  fathomed  by  bound,   it  is  continued.     Basically, 
for  each    i    _f  j(x),  a  new  correction  of  the  form  x  +  e     is  generated.     This 
correction  is  tested  for  bound  by  comparing 

Zq+c.  +G(3-    _Z^ajXj   -   a^) 

with  the  incumbent  cost.  If  the  bound  for  this  new  correction  is  less  than 
t,   the  group  network  is  used  to  backtrack  this  correction.  If  x  +  e.  +  x 
is  feasible,  then  a  new  incumbent  has  been  found,  and  it  is  recorded  along 
with  its  costs.  (Parenthetically,  we  note  that  by  maintaining  B  and  a. 
cleared  by  the  determinant,  the  system  can  use  integer  arithmetic  to  test 
feasibility  in  (4b).  Integrality  in  (4b)  is  guaranteed  through  the  use  of  the 
group  problem. ) 

If  the  new  continuation  is  not  feasible,  then  it  is  saved  on  the  sub- 
problem  list  for  later  consideration. 

In  fact  the  search  routine  does  not  generate  all  continuations  x  +  e., 
j  ^  j(x)  for  a  given  x.  Recall  that  the  backtracked  solution  from  the  group 
network,  x  ,  is  stored  with  the  subproblem.  It  can  be  shown  that  any 
correction  x  +  u  ^x  +  x  will  be  infeasible  if  x  +  x  is  infeasible,  and 
moreover  the  former  will  be  backtracked  to  x  +  x  .  Hence  only  descendants 
X  +  u  ^  X  +  X  are  created  and  tested. 

When  all  these  descendants  of  x  have  been  created,  tested,  and  saved  if 
necessary,  the  subproblem  x  is  deleted.  This  is  possible,  of  course,  because 
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fathoming  the  descendants  of  x  is  equivalent  to  fathoming  x. 

The  particular  search  routine  in  our  system  is  of  the  most  straightforward 
kind.  As  can  be  seen  from  Table  4,  it  can  generate  and  test 

solutions  Mery   rapidly.  On  the  other  hand,  the  real  issue  is  how  rapidly 
a  problem  can  be  solved.  Finding  a  good  incumbent  early  can  have  a  dramatic 
effect  on  the  efficiency  of  the  search.   The  current  algorithm  does  not  give 
sufficient  emphasis  to  this  goal.  The  theory  for  an  improved  search  is 
presented  in  (4)  and  we  hope  to  implement  it  along  with  some  heuristics  for 
managing  the  search. 


This  is  the  reason  we  backtrack  solutions  as  they  are  generated  in 
the  hopes  of  finding  an  incumbent  early  in  the  search. 
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Table  4. 


Search  Times 


Number  of 

Time 

Rows 

Non-basics 

Corrections 

(sec) 

Machine 

1 

12 

104 

99 

0.77 

1108 

2 

9 

12 

6769 

7.14 

360/85 

3 

36 

36 

7979 

103.20 

360/67 

4 

9 

14 

8463 

19.63 

360/67 

5 

12 

104 

11570 

67.68 

1108 

6 

9 

14 

12814 

7.51 

360/85 

7 

14 

13 

20692 

10.51 

360/85 

8 

12 

104 

169902 

253.00 

1108 

9 

313 

170 

668420 

361.85 

360/85 
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Concluding  Remarks 

We  have  tried  to  demonstrate  in  this  paper  that  group  theoretic  methods 
are  easy  to  derive  and  effective  in  solving  IP  problems.  It  is  important 
to  reemphasize  that  the  algorithm  IPA  which  uses  these  methods  is  perhaps 
the  simplest  one  possible.  After  finding  an  optimal  LP  basis,  IPA  derives 
one  group  problem,  either  UGP  or  ZOG,  solves  it,  and  then  if  necessary, 
implicitly  searches  the  set  of  all  non-negative  corrections  where  trial 
corrections  and  the  extent  of  the  search  are  derived  from  the  previously 
solved  group  problem.  In  spite  of  its  simplicity,  IPA  has  performed  quite 
well  on  a  variety  of  real  life  IP  problems  as  indicated  in  the  figures  in 
Table  1.  We  believe  that  future  algorithms  will  be  even  more  effective  when 
we  implement  the  dual  methods  of  [4]  for  improving  the  bounds  and  generati  ig 
strong  cuts,  and  when  we  make  dynamic  use  of  the  group  optimization  prob- 
lems as  discussed  in  [12].  A  third  important  area  of  improvement  and 
innovation  is  the  use  of  data  manipulation  for  controlling  basis  determinaits 
by  the  methods  discussed  in  [13].  We  will  conclude  this  paper  with  a  dis- 
cussion of  the  importance  of  the  basis  determinants  in  IP  computation. 

Consider  the  313  row  airline  crew  scheduling  problem  in  Table  1.  When 
the  optimal  LP  basis  obtained  by  MPS  was  pivoted  into  an  identity  matrix 
in  preparation  for  solving  the  problem  as  an  IP  problem,  the  determinant 
equalled  +1  or  -1  for  311  pivots  and  then  went  to  24  on  pivot  312  and  48 
on  pivot  313.  Thus,  the  LP  basis  was  almost  unimodular  and  this  made  the 
problem  easy  to  solve  in  spite  of  its  large  size.  IPA  found  good  solutions 
to  a  more  difficult  airline  crew  scheduling  problem  that  had  259  rows  and 
about  750  variables;  it  is  not  listed  in  Table  1  because  we  were  not  able 
to  time  its  performance  except  to  estimate  the  total  solution  time  at  about 
15  minutes  on  an  IBM  360/65.  This  problem  had  a  determinant  of  176  and 
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this  solution  was  very  fractional.  It  is  interesting  to  note  that  we  found 
alternative  optimal  bases  for  this  problem  with  determinants  of  8  and 
103,000,  so  the  selection  of  an  optimal  basis  is  important.  Thiriez  [22] 
also  reports  on  the  successful  use  of  group  theoretic  methods  in  airline 
crew  scheduling. 

By  contrast  to  the  313  row  problem,  consider  the  difficult  26  row 
zero-one  problem  (application  unknown).  Other  codes  had  been  tried  on 
this  problem  with  little  success.  The  coefficient  matrix  of  this  problem 
consisted  of  O's,  1 's  and  2' s,  and  the  determinant  of  the  optimal  LP  basis 
was  on  the  order  of  5000.  Another  problem  of  this  class  with  an  additional 
less  than  or  equal  to  constraint  consisting  of  small  positive  integers 
with  a  right  hand  side  of  235  had  an  optimal  basis  determinant  of  155,000, 
We  relaxed  the  constraint  by  the  methods  of  [13]  and  hoped  to  satisfy  it 
a  fortiori  during  search.  During  one  run  we  generated  37  solutions  which 
were  feasible  on  all  of  the  constraints  except  the  indicated  one,  but 
violated  the  relaxed  one  by  approximately  10  to  20  units  out  of  235. 
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